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ABSTRACT 

(.y  ■’  :  ■'  ,  '  ' 

Given  a  grammar  G  and  a  string  a,  all  possible  parses 
of  o  can  be  constructed  by  repeatedly  applying  the  rules  of 
G  in  parallel.  This  process  creates  a  "lattice  graph"  in 
which  any  directed  path  from  the  least  element  to  the  great¬ 
est  element  is  a  sentential  form  that  occurs  in  a  (partial) 
parse  of  a.  Examples  are  given  illustrating  how,  at  least 
for  some  grammars,  this  process  does  not  lead  to  a  combina¬ 
torial  explosion,  and  could  thus  be  used  to  parse  strings 
very  rapidly  if  suitable  parallel  hardware  were  available. 


% 


i  >  . 


The  support  of  the  U.S.  Air  Force  Office  of  Scientific 
Research  under  Grant  AFOSR-77-3271  is  gratefully  acknowledged, 
as  is  the  help  of  Sherry  Palmer  in  preparing  this  report. 


Introduction 


1 . 

Let  G  be  a  grammar,  which  we  shall  first  assume  for 
simplicity  to  be  context-free,  with  rules  of  the  form  A-*-a 
(a  non-null)  .  Conventionally,  to  parses  a  string  a  with 
respect  to  G,  we  find  a  match  in  o  to  the  right-hand  side 
(RHS)  of  some  rule  A+a;  replace  this  instance  of  a  by  A; 
and  repeat  the  process,  until  0  is  reduced  to  a  single  S 
(the  "start  symbol”  of  G) . 

Time  bounds  on  context-free  parsing  have  been  extensively 

studied  (e.g.,  [1]).  For  general  context-free  grammars, 

the  time  required  to  parse  a  string  of  length  n  is  on  the 
3 

order  of  n  ;  and  for  non-context-free  grammars  the  situa¬ 
tion  is  presumably  worse. 

One  could  image  parsing  "in  parallel"  by  replacing 
many  cu  ' s  by  A^'s  simultaneously,  but  this  leads  to  diffi¬ 
culties  if  the  RHSs  overlap;  for  example,  if  A-*-a  and  B-»-3 
are  rules,  where  3  is  a  substring  of  a,  where  do  we  put  the 
B  relative  to  the  A  when  we  apply  both  rules?  In  [2]  it  is 
suggested  that  parallel  parsing  can  be  done  by  a  two-step 
process,  first  choosing  nondeterministically  which  RHSs 
should  be  rewritten,  and  then  actually  rewriting  them  iff 
no  overlapping  RHS  was  chosen  for  rewriting.  It  is  shown 


in  [2]  that  the  language  parsed  in  this  way  is  the  same  as _ — 1 

the  language  of  G  as  ordinarily  defined.  1  s 

1 

n 

■c 

This  note  describes  an  alternative  approach  in  which 
we  actually  rewrite  all  RHSs  in  parallel,  and  represent 
the  result  not  by  a  string  but  by  a  "lattice  graph"  in 
which  paths  between  the  terminal  points  correspond  to 
sentential  forms.  It  is  easily  seen  that  this  process 
yields  all  possible  parses  of  the  given  string.  Examples 
are  given  illustrating  how,  at  least  for  some  grammars, 
this  process  does  not  lead  to  a  combinatorial  explosion. 
Implementation  of  this  approach  using  a  reconf igurable 
network  of  processors  is  also  briefly  discussed.  Such  an 
implementation  would  permit  parsing  to  be  carried  out 
rapidly,  with  the  time  required  depending  primarily  on 
the  height  of  the  parse  tree. 


Let  G  be  an  acyclic  directed  graph  with  set  of  nodes 
N.  If  there  is  a  (directed)  path  from  p  to  q,  where  p,q 
are  in  N,  we  say  that  p^q.  Evidently,  s  is  a  partial  order 
relation:  reflexive  (p-p  for  all  p) ,  antisymmetric  (P-q  and 

q5p  imply  p=q) ,  and  transitive  (p-q  and  q5r  imple  p^r) .  We 
say  that  g  is  the  greatest  lower  bound  (gib)  of  a  set  of  nodes 
N'cN  if  g£p  for  all  p£N,  and  g'Sp  for  all  p€N'  implies 
g'Sg.  Similarly,  we  say  that  Z  is  the  least  upper  bound 
(lub)  of  N'  if  p^£  for  all  p6N’,  and  p5£'  for  all  p€N' 
implies  Z^Z'.  We  call  G  a  lattice  graph  if  every  nonempty 
N'cN  has  a  gib  and  an  lub.  In  particular,  the  gib  and  lub 
of  N  itself  exist;  we  denote  them  by  0  and  1,  respectively. 
Note  that  there  is  a  path  from  0  to  any  p€N,  and  from  any 
p€N  to  1 ,  so  that  G  is  connected.  If  (p,q)  is  an  arc  of 
G  (so  that  p^q) ,  we  call  p  a  predecessor  of  q  and  q  a  suc¬ 
cessor  of  p. 

Any  string  xo'*-Xn  may  be  regarded  as  an  acyclic 
directed  graph,  with  arcs  between  successive  symbols  (x^_^,x^) 
l'in.  Evidently  a  string  is  a  lattice  graph.  We  shall 
assume  that  the  given  string  a  which  is  to  be  parsed  begins 
and  ends  with  endmarkers,  say  $crC;  thus  $=0  and  £=1  when 
we  regard  the  string  as  a  lattice  graph. 

Let  a  be  a  substring  of  a,  and  let  A->-a  be  a  rule  of 
the  grammar  G,  where  ct  =  x^...Xj  (say).  When  we  apply  this 
rule  to  o,  we  create  a  "short  cut"  through  A  from  the 


j 

i 


predecessor  x^_^  of  a  to  its  successor  Xj+^: 

A 

• ’ -xi-ixi- *  *xjxj+l* ' * 

(Here  the  bar  extends  from  just  after  the  precedecessor  of 

A  to  just  before  its  successor.)  Evidently  the  result  is 

still  a  lattice  graph,  though  it  is  no  longer  a  string. 

The  situation  is  analogous  if  we  apply  many  rules  to  a 

simultaneously,  even  if  their  RHSs  overlap;  we  still  obtain 

a  lattice  graph,  e.g. 

A 


.  w. 


B 


Note  that  any  directed  path  from  $  to  C  through  this  graph 
represents  a  possible  sentential  form  derivable  from  o  using 
the  grammar  G. 

After  the  first  round  of  (parallel)  rule  application,  we 
no  longer  have  a  lattice  graph  that  is  a  string,  but  any 
directed  path  from  $  to  f  is  a  string,  and  we  can  still  apply 
rules  to  the  lattice  graph,  by  matching  their  RHSs  against 
all  possible  substrings  of  these  strings.  Such  applications 
create  further  "short  cuts"  in  the  graph,  but  it  remains  a 
lattice  graph.  If  we  ever  create  a  short  cut  in  which  we 
can  go  directly  from  $  to  C  via  a  single  S  (the  start  symbol 
of  G) ,  we  have  successfully  parsed  a. 

To  clarify  these  ideas,  we  give  a  simple  example.  Let 
G  be  the  parenthesis  string  grammar  whose  rules  are 


S-SS  S-(S)  S-M) 


and  let  o  be  the  string  $(()()  ())()£ 


In  the  first  round  of  parallel  parsing,  the  rule  S+()  applies 
in  four  places,  yielding  the  graph 


$  (  0  (_)  0  )  (_)  $ 

S1S1S1  S1 


where  the  subscript  indicates  the  round  number.  Note  that 
none  of  these  rules  have  overlapping  RHSs.  In  the  second 
round,  there  are  many  possible  paths  from  $  to  C ,  e.g. 
$(S()S)()C,  but  the  only  ones  that  allow  new  rule  appli¬ 
cations  are  those  that  contain  two  or  more  consecutive  Ss. 
Thus  the  second  round  yields 


$  ( (_)  U  (_)  )  (_K 
S1S1S1  S1 


where  the  two  rules  now  do  have  overlapping  RHSs.  At  the 
third  round,  we  still  have  new  rule  applications  for  the 
paths  that  contain  two  consecutive  Ss  -  i.e.,  the  paths 
for  which  the  first  two  or  the  last  two  Ss  were  rewritten, 
but  not  all  three;  but  both  of  those  paths  yield  the  same 
new  path: 


The  fourth  round  thus  allows  us  to  rewrite  (S,)  as  S., 

3  4 

and  the  fifth  round  to  rewrite  as  S^,  completing 


the  parse  of  a : 


$  ( (J  y  (_) )  (_)  <f 

slslsl  S1 


In  this  example  there  is  an  ambiguity  (the  two  S2's) 
in  which  both  alternatives  lead  to  the  same  result,  but  there 
are  no  "dead  ends"  (rule  application  sequences  which  do  not 
lead  to  a  parse) .  As  another  example,  consider  the  grammar 
for  palindromes  of  even  length  whose  rules  are 


S-*aa 


S+bb 


S+aSa 


S+bSb 


and  let  a  be  the  string 


$aaabbaaa<? 

Here  at  the  first  round  we  have 

*_1  fl 

$aaabbaaa<? 

S1  S1  S1 

where  only  the  center  S  is  part  of  a  correct  parse.  In  fact, 
at;  the  next  round  the  other  four  Ss  do  not  contribute  to  rule 
applications;  only  the  center  one  gives  us 

fl  fi 

$aaabbaaa<? 

S1  S1  S1 


and  similarly  at  successive  rounds  we  rewrite  aS2a  as  S3  and 
aS^a  as  S^,  completing  the  parse  of  a. 


A  third  example  uses  the  following  grammar  for  the 


set  of  strings  that  contain  equal  numbers  of  a's  and  b's: 
S  ■*  aBS  or  bAS  or  aB  or  bA 
A  -*■  bAA  or  a  ;  B  -*■  aBB  or  b 
For  the  string  aabaabbb  we  have  the  following  parallel 
parse : 


B, 


$  a  a  b  a  a  b  b  b 
A^  A^  B^  A^  A^  B^  B^  B^ 


B . 


C 


The  reduction,  rather  than  increase,  in  the  symbols  created 
at  each  step  is  apparent:  8  at  stage  1,  5  at  stage  2 ,  3  at 
stage  3,  2  at  stage  4,  and  1  (the  parse  completion)  at 
stage  5. 


i 


3.  Complexit 


The  parallel  parsing  process  described  in  Section  2 
constructs  all  possible  (partial)  parses  of  o,  even  mutually 
inconsistent  ones;  indeed,  after  n  rounds  of  parallel  rule 
applications,  every  possible  sequence  of  Sn  rule  applica¬ 
tions  is  represented  by  a  path  from  $  to  C  in  the  lattice 
graph.  Thus  if  a  has  a  parse  of  length  n,  the  path  $S<£  will 
occur  in  the  graph,  and  indeed  every  sentential  form  in  the 
parse  will  also  occur  as  a  path. 

Of  course,  parallel  parsing  can  potentially  lead  to 
a  combinatorial  explosion.  Even  if  the  graph  itself  does 
not  become  very  large,  the  number  of  paths  from  $  to  C  in 
the  graph  will  grow,  and  these  paths  must  (in  principle) 
all  be  checked  at  each  round  for  possible  new  rule  applica¬ 
tions.  However,  if  the  number  and  length  of  the  rules  are  not 
too  large,  the  amount  of  checking  required  cannot  be  very 
large.  If  the  average  (out)  degree  of  a  node  of  the  lattice 
graph  is  d,  the  average  number  of  strings  of  length  k  that 
start  at  a  given  node  is  d  ;  thus  if  all  rule  RHSs  have 
length  ^k,  we  need  only  check  d  possibilities  for  each 
node,  on  the  average.  Moreover,  it  should  be  possible  to 
use  fast  string  matching  techniques  to  reduce  the  amount 
of  checking  that  is  needed.  The  examples  given  in  Section 
2  suggest  that  at  least  for  some  grammars,  the  graph  does 


A 


not  grow  rapidly,  and  multiple  paths  resulting  from 
alternative  choices  at  a  given  stage  may  recombine 
(i.e.,  lead  to  the  same  derived  path)  at  a  later  stage. 


A  possible  approach  to  reducing  the  combinatorial 
growth  might  be  to  apply  only  a  subset  of  the  rules  that 
are  applicable  at  a  given  stage,  where  the  subset  is 
chosen  on  heuristic  grounds  as  being  somehow  "most  likely" 
to  lead  to  a  parse  -  e.g.,  apply  the  rules  whose  RHSs  are 
longest  (so  that  they  yield  the  shortest  sentential  forms) , 
or  the  rules  that  lead  to  nonterminal  symbols  that  are 
derivable  from  the  start  symbol  in  the  fewest  possible 
steps.  However,  it  is  easy  to  contrive  grammars  in  which 
such  heuristics  would  not  lead  to  a  successful  parse. 

Another  possibility  might  be  to  apply  all  possible  rules 
at  a  given  stage,  but  then  allow  the  results  to  participate 
in  a  cooperation/competition  process  (e.g.,  rules  that  give 
rise  to  overlapping  parts  of  a  compete,  since  they  are 
mutually  inconsistent;  if  a  rule  creates  a  symbol  used  by 
a  later  rule,  the  latter  reinforces  the  former) ,  and 
eliminate  rule  applications  that  have  too  much  competition 
and  not  enough  support.  Here  again,  however,  it  is  not 
hard  to  contrive  examples  in  which  this  would  eliminate 
rules  that  are  necessary  for  a  parse.  The  use  of  heuristics 
for  rule  selection,  and  cooperation/competition  ("relaxation") 
for  rule  elimination,  will  therefore  not  be  investigated  here. 


4 .  Implementation 

Tne  implementation  of  the  parallel  parsing  process  is 
straigntforward ,  and  does  not  require  explicit  construc¬ 
tion  or  the  lattice  graph.  Our  method  of  implementation  is 
based  on  the  fact  that  for  context-free  grammars,  any  sym¬ 
bol  appearing  in  a  parse  has  a  well-defined  "scope"  with 
respect  to  the  original  string  a,  i.e.,  it  arises  from  a 
specific  substring  of  a.  Moreover,  two  symbols  occur  con¬ 
secutively  on  a  path  in  the  lattice  graph  iff  their  scopes 
are  consecutive  substrings  of  a.  Based  on  these  observations, 
given  the  string  $o£=  $Xg...xC,  we  create  for  Oiisn  a  list 
of  the  symbols  whose  scopes  begin  in  the  ith  position,  and 
for  each  of  these  symbols  we  give  the  position  at  which  its 
scope  ends.  Initially,  consists  of  x^  alone,  with  ending 
position  i.  To  find  the  paths  through  the  graph  that  begin 
with  a  particular  symbol  A  (say  on  list  ,  with  ending  posi¬ 
tion  j),  we  use  the  fact  that  the  successors  of  A  on  all  such 
paths  are  just  the  symbols  on  list  Lj  +  ^,  and  we  repeat  this 
process  to  find  the  subsequent  symbols. 

To  apply  a  new  rule,  say  A-*-B1B2 .  . .  BK,  we  proceed  as  follows 
Scan  the  lists  for  all  occurrences  of  B^.  For  each  occurrence, 
go  to  its  successor  list  and  check  for  the  presence  of  B^;  for 
each  of  these,  go  to  its  successor  list  and  check  for  B^;  and 
so  on.  If  the  rule  has  a  short  RHS,  the  number  of  possibili¬ 
ties  to  be  checked  should  not  be  very  large.  If  we  find  a 

match  to  the  entire  RHS,  say  with  B.  on  list  L.  and  with  Bv 

-i-  1  K 


having  ending  position  j,  we  add  an  A  to  list  with  ending 
position  j.  Note  that  two  rule  applications  may  lead  to 


the  same  result,  if  there  are  two  sequences  B^,...,BK  in  which 
the  B, 's  are  on  the  same  list  L.  and  the  B  ' s  have  the  same 

X  1  I\ 

ending  position  j;  in  fact,  two  rules  A-*-B^...,BK  and  A-»-C....,CH 
may  also  lead  to  the  same  result  if  B^ ,  are  both  on 

and  B  ,  C  both  end  at  j.  After  each  round  of  rule  applications, 

K  H 

we  should  check  each  list  for  duplicates  (same  symbol 
with  same  ending  position)  and  eliminate  them.  If  we  want 
to  maintain  strict  parallelism  in  applying  the  rules,  the  A's 
that  we  find  should  be  put  on  a  separate  set  of  lists  1/  , 
rather  than  on  the  current  lists  L^;  when  we  have  completed 
a  round  of  rule  applications,  we  append  each  1/  to  the  corre¬ 
sponding  and  check  for  duplicates. 

Parallel  rule  application  could  be  implemented  by  a  multi¬ 
processor  system  as  follows:  Each  processor  maintains  one 
of  the  lists  (or  several,  if  there  are  more  symbols  in  the 
initial  string  than  there  are  processors).  To  apply  the  rule 
A-*-B^ .  . .  Bj, , ,  we  broadcast  it  to  all  the  processors.  Any  pro¬ 
cessor  having  B^'s  on  its  list  sends  messages  to  the  processors 
responsible  for  the  successor  lists  to  check  for  B2's;  and  so 
on.  If  a  sequence  goes  to  completion,  the  processor  that  had 
the  corresponding  B^  on  its  list  adds  an  A  to  .  Note 
that  the  amount  of  time  required  for  parallel  application  of 
A"B^...BK  is  not  simply  proportional  to  K,  since  many  messages 
may  arrive  at  the  same  processor  simultaneously  and  must  then 
wait  to  be  processed. 


5. 


rammars 


Generalization  to  arbitrary  g 

Our  parallel  rewriting  process  extends  in  principle  to 
grammars  that  are  not  context-free,  but  there  are  some  com¬ 
plications.  Given  a  rule  a+3 ,  we  can  apply  it  by  construct¬ 
ing  a  "bypass"  (not  necessarily  a  short-cut,  since  a  may  be 
longer  than  3)  in  which  3  follows  the  predecessor (s)  of  a 
and  is  followed  by  the  successor (s)  of  a.  If  some  of  the 
symbols  in  a  and  3  are  the  same,  e.g.,  in  the  context-sensitive 
rule  it  would  be  more  economical  to  construct 

bypasses  only  for  the  new  symbols,  i.e.,  A  follows  the  pre¬ 
decessors  of  a  and  is  followed  by  its  successors.  Note, 
however,  that  this  could  give  rise  to  paths  from  $  to  C  that 
could  never  occur  as  sentential  forms. 

To  illustrate  how  the  process  might  work  in  a  non-context- 
free  case,  we  give  a  context-sensitive  example.  The  language 
{anbncn  |  nr:l}  has  the  grammar 

S-*-abc  S-*-aTBc 

T+aTBC  T+abC 

CB+BC  bB-*-bb  Cc+cc 

For  the  string  aaabbbccc  we  have  the  following  parallel  parse: 


a 


CjC 


c 


B. 


C 2  c 


B. 


$  a  a 


c  C 


Note  that  bB^  and  C2  c  have  the  same  positions  as  bB^  and  C^c, 
but  are  not  duplicates;  B2  has  the  other  B^  as  a  successor, 
and  has  the  other  as  a  predecessor.  Similarly, 
has  the  same  position  as  C2B2,  not  a  ^Pl^-03*-6  (i-n  fact 

it  arose  from  B^  and  C2,  which  were  produced  only  by  rewriting 
C2  and  B2) ;  e.g. ,  C2has  b  and  as  predecessors,  but  C4  does 
not  (fact,  its  sole  predecessor  is  C^»  which  arose  from  re¬ 
writing  3^)  . 

We  see  from  these  remarks  that  the  simple  implementation 
given  in  Section  4  for  the  context-free  case  does  not  general¬ 
ize  to  the  context-sensitive  case.  Rather,  it  becomes  neces¬ 
sary  to  construct  the  lattice  graph  explicitly,  with  pointers 
from  each  symbol  to  its  successors.  The  graph  is  likely  to  be 
bigger,  and  it  becomes  much  more  difficult  to  detect  duplicate 
paths  in  the  graph. 


f 
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G .  Concluding  remarks 

With  the  increasing  availability  of  highly  parallel 
hardware,  a  parallel  approach  to  parsing  may  deserve 
serious  consideration  .  The  effectiveness  of  this  approach 
depends  on  limiting  the  combinatorial  growth  of  the  parse 
graph,  but  in  some  cases  this  growth  may  not  be  excessively 
explosive.*  If  the  strings  to  be  parsed  are  not  too  long, 
parallel  hardware  is  available,  and  processing  time  is  a 
significant  consideration,  parallel  parsing  becomes  an 
attractive  alternative  to  conventional  sequential  methods. 


*  Experimental  studies  of  the  growth  rate  of  the  graph  for 
various  types  of  grammars  are  planned. 
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